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Overview 


Fault trees represent problem situations by organizing 
"things that could go wrong" into functional categories. Such 
trees are essential devices for analyzing and evaluating the 
fallibility of complex systems. A series of six experiments 
found that the perceived fallibility of a system and its 
several conponents is extremely sensitive to subtle aspects of 
the way in which the system is presented. Implications for 


systems analysis and control are discussed. 


Background 


Many problems involve some form of troubleshooting: 
something goes wrong and the problem solver attempts to figure 
out why by first listing and then checking possible causes. 

A common representation for such probiems is a "fault tree," 
which organizes possible sources of trouble into a branching 
structure. Once constructed, the tree serves as a guide for 
the problem solver who might ask: Which system is more likely 
to be the source of trouble? What information would allow me 
to check out the fault most quickly? How well do I know this 
system (i.e., how authoritative is the fault tree) and should 
I be tinkering with it myself? Fault trees are used .ot only 
in analyzing systems that have gone astray, but also in 
attempting to design fail-safe systems. Knowing how things go 
wrong is a prerequisite to drafting directives on what to do 
right. As with many other kinds of problems, any trouble- 


shooting situation may be represented in a variety of ways. 


ia 


Approach 


The studies reported here look at the impact of three 
such "arbitrary" aspects of the way in which fault trees are 
presented on the way in which they are evaluated. These are: 


(a) what is listed specifically and what is left to "all other 


problems," (b) how much detail is presented for the various 
"branches" (systems) of the fault tree, and (c) how various 
systems are grouped into branches. The fault tree used 


detailed the various ways in which a car might fail to start. 
The fault trees studies has four to eight branches, including 
"battery charye insufficient," "fuel system defective" and "all 


other problems." 


Findings 


Major results were: (a) people were quite insensitive 
to what had been left out of a fault tree, (b) increasing the 
amount of detail for the tree as a whole or just for some of 
its branches produced small effects on perceptions, and (c) the 
perceived importance of a particular branch was increased by 
presenting it in pieces (as two separate component branches). 
Insensitivity to omissions was found both with college student 


subjects and experienced garage mechanics. 

Implications 

When technical experts attempt to present risk information 
to non-experts with decision-making responsibility, subtle 


aspects of their presentation may have sizeable consequences for 


the way in which those risks are perceived. Only a detailed 


id 2. 


understanding of these consequences can insure that decision 


makers are properly informed. Some specific implications and 
p 


procedures are discussed. 


Also discussed is the possibility that technical experts 
are subject to similar biases when they structure their analyses. 
Some anecdotal evidence suggests that this may be the case and 


points to some specific problems to be avoided. 
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1. INTRODUCTION 


Many problems involve some form of troubleshooting: 
something goes wrong and the problem solver attempts to figure 
out why by first listing and then checking possible causes. 

The problem solver's representation of the problem (and possible 
causes) may be constructed from scratch, retrieved from memory 
or adopted from an external source, perhaps with supplementary 
information from memory. A common representation for such 
problems is a "fault tree," which organizes possible sources 

of trouble into a branching structure. Figure 1-1 presents a 
fault tree for the event "a car fails to start long enough to 
delay the driver for one minute." The top row presents the 
problem and the next row indicates the major systems whose 
failure might be relevant. Below each major system are listed 
specific contributing failures. Sources employed in 
constructing this particular tree were Petersen's Basic Auto 
Repair Manual, The Dodge Colt Service Manual, The Chevelle 
Owner's Guide and several experienced mechanics. Using 
analogous sources of expertise, one could con..truct a tree 
headed by "three-month-old infant cries more than five minutes," 
"bank statement does not match checkbook," "star pitcher fails 
to report for spring training," "SAC bomber recall system fails" 


or nuclear power reactor core melts." 


Once constructed, the tree serves as a guide for the 
problem solver who might ask: Which system is more likely to 
be the source of trouble? What information would allow me to 
check out the fault most quickly? How well do I know this 


system (i.e., how authoritative is the fault tree) and should 


I be tinkering with it myself? Fault trees are used not only 


Sweavene 


wana ate 


Darwim 10 NOS2 S10 
5D 10 DNINONA ¢ 

bAWALiWU 9 Tames Bw dA 
40 IDvav IMs wORaIL 


ONE Feel A add 
Sa ty SIMA ae 


LUVLS L,NOM YVO W AHM 
ONIYMHAOOSICG 8Cl AHAL LIAWA ATAISsod W 


| INDDUMSNIGAL 


WOOL ues A 4D WEIS oP 


dxOH2 O82 dwlWs 
S423 dywantaawa £ 
N4s0 FATWA BxOMD 4 


90200342 WO NMOOHSUM SwD 
3 ROE WO tO BION I AD 
WO SON NOISE NOISE 
DWINDUS WO OREOW 


AWeGUat ANENE FATWA 
OW S98 9009.49 FRIED 
2 1nS wD Ova: Seva) 


1S0ew2 wOt INO 5) BOL eNOUT Ss 


200034 avwie 1303 Husd2ta wv 
Ave 100 

WO MEATS Us 11s SS 19KE 

eDiwe bite 

90 ON avel es SSVI D 

Amar sncow 10 1NG AWUTs 

BA IWA 297 Deinw dd 

MUthe O02 PUNSS Fu ata 14H] 


220U0 URDU MM DRIES SONIS © 
Onetiias 241234590 WO BOOT 
DMIBUME ON 
079802 60 13 eHow awd 


cmneia 


38007 wo O49NGS 
QsDwHVO TAD UD ONY 
AV aH S382 

One NONCER 


an 


as007 
29mS99 OW sieve PuWLND 
V0OG WOL tee DIN DIED 
MWe 194 940239900 

SPA TWA O Siw AS A Te ewan 
Naeows Wad Tans 


Ae ss00d O9A5R 106 WOLON 8 
940 Dra 


a 
82Oustavers 8 

wiK ONLI ALIOvA « 
IlwwLSISJy AIS HUI» 
40 1NIOd wieOnure 
O4OUwOD StuOHIIIIa 
aisowudev 4 


Pet) VE OE VOID 


002 001 wanivin Sw2 91M ww? 


Was ONO 


SAN ANON? 
Oe} AAG Ms DPasetian 100239500 0 
NOISS IU IOI WOODS Ag 24210 SONIA mews 
200s SHOE aainvs wOrimessIO & 
wot, OO1 WO aLWws W026 


Seva tee 
sesund ewer 


(05209 iw TAL I1I 


*UPNDOO V4 VIS SIDED 
VaMD ANIA 


gabe aoe ie 
miasan bane 


“T-{ wand 


eed Sirs 
BdAINO 2402 0 Uw bL8wS 1190 | 
Od 2457 om £320 ase bert 
46004 RO 439780 AAU IN 
3A1439530 
21 aad 2328 Bw RAINED 
Gandstws 10MELI98 | 


O10N 2108 

AILS sewss Dw dn de 
avlie whiewss 
HD4tAab NOL LODE 


3A1893438 JAW whduwEs 
SOLON WALNwas ALIAS 
Siw? 

4G8) Vs ROS 2970 bv 3S 
Wwainin wo 

Min ve te LOM NOISES UL 
3A18 23630 536 341MS 


Vewes Gos aueiive 

9 amen) 0201 

C0 vee pens ee Pee we 
feree Lheem Some 10D 

Sow eret wets 

CmueeteD 
SMUOS Ese sees he aewe OOS 
AAD eed Yemen! Tae 
DALI I¢ em eGaw wrod Bee TOR 
2d one te 

SP eOw Byes F wD 

eheore 

OG pT Gets Ape ee 28 
sae 4 deed ae S001 
POer aie tD be 2ds20 


sec ae hed eR 


$0015 3 20002 25001 
awed 


MvimAURIIWe © 
@s00uH02 HO 
9300) Stwinda £ 
W011 INNOD 
GnNOUD ALWiws 2 


AG He 4 APs 
saws Aw asa ee 


i=2 


Saeaiiacteabin._ail 


ae is. 


in analyzing systems that have gone astray, but also in 
attempting to design fail-safe systems. Knowing how things 

go wrong is a prerequisite to drafting directives on what 

to do right. Fault trees have been instrumental in the design 
of technological systems from space ships to nuclear power 
plants (Atomic Energy Commission, 1975; Bryan, 1974; Green & 
Bourne, 1972). 


Fault trees are also used to estimate failure rates 
for complex systems when historical data for the system as 
a whole are unavailable. Probabilities are assigned to each 
of the pathways to failure and then combined to provide an 
overall failure rate. Such analyses were the primary 
wethodological tools in the $3 million Rasmussen study 
assessing the probability of a catastrophic loss-of~-coolant 
accident in a nuclear power reactor (Atomic Energy Commission, 
1975). Fault-tree analysis has, however, come under attack 
from critics who question whether it is methodologically 
sound enough to be used as a basis for decisions of great 
consequence (e.g., Bryan, 1974). One major concern of the 
critics is that omission of relevant pathways due to ignorance, 
poor memory or lack of imagination would lead to an 


underestimation of failure probabilities (Kendall, 1975). 


As with many other kinds of problems (Newell & Simon, 
1972), any trouble-shooting situation may be represented in a 
variety of ways. For example, in constructing the fault tree 
in Figure 1-1, it was necessary to decide how much detail to 
provide for each system; whether to present minor systems, like 
"mischievous acts," separately or to lump them with "all other 
problems"; whether the four items grouped in "fuel system 
defective" actually belong together or whether the last two 


might best be listed separately with a heading like "carburetion 
problems"; whether to use the graphic display or an outline; and 
whether to use this level of specificity, or more, or less. 
Making these decisions will depend upon considerations like: 

the purpose of the analysis, the amount of knowledge available 
about the issue in question, and how much of an effect each of 
these aspects of presentation has upon the way people evaluate 


the system. 


The studies reported here look at the impact of three 
such "arbitrary" aspects of the way in which fault trees are 
presented on the way in which they are evaluated. These are: 
(a) what is listed specifically and what is left to "all other 
problems," (b) how much detail is presented for the various 
"branches" (systems) of the fault tree, and (c) how various 


systems are grouped into branches. 


Aside from their interest for students of problem 
solving, such questions may be important for those concerned 
with the management of risks in our society. The lay public 
is increasingly called upon to decide whether the risks from 
various technological systems are acceptable in the light of 
the accompanying benefits. The risks from many proposed 
projects (e.g., nuclear power, liquid natural gas, recombinant 
DNA research) are varied and complex. Typically, they are 
presented to the public by technical experts who in one way or 
another utilize fault-tree representation. In preparing their 
presentation, they must make arbitrary decisions like those 
listed above. Understanding what difference those decisions 
make might help us understand (a) whether the public is being 
properly informed (e.g., is risk information being presented 


in ways leading to its subjective overestimation or 


underestimation?), (b) how technical information can be 
communicated so that it is perceived most veridically, and 

(c) what possibilities exist for manipulating perceptions 
through judicious fault-tree presentation. If the perceptions 
of experts are also affected by variations in tree 
representation, we might learn something about how to improve 
the basic methodology of a technique used for momentous 


decisions. 


Experiment 1: Pruning the Tree 


However exhaustive one would like to be in the design 
of a fault tree, at some point it is necessary to stop listing 
alternatives and combine the remainder into a category labelled 
something like "all other problems." In theory, listing every 
possible cause is an endless chore, and in practice, listing 
many implausible causes could overwhelm the fault tree's 
designers and viewers with bizarre possibilities that might 
divert attention from the primary possibilities. In 
Experiment 1, we studied the sensitivity of an evaluator to 
fault-tree components that have been omitted. In this initial 
investigation, we looked for gross effects, those obtained by 
deleting substantial portions of a tree without specifically 
indicating their absence. In the real world of design and 
disaster, elements can be left out either inadvertently (the 
designer lacks the appropriate knowledge or imagination) or 
consciously (the designer collapses certain categories into 
"other" category for simplicity; the designer omits certain 


pathways in order to make the system appear safer). 


Two contrary hypotheses may be advanced for the impact 
of deleting major components of a fault tree on a viewer's 


judgment of its completeness. 


One hypothesis suggests that when major items are 
deleted, it is quite likely that the absence of at least one 
wils be detected. Once such an omission has been uncovered, 
the entire analysis is discredited and the proportion left 
out exaggerated. (Imagine your reaction to discovering that 
a purported fault tree for "car won't start" contains no 
mention of battery failure.) Many public discussions regarding 
nuclear power and other technological risks would seem to 
support this hypothesis. Members of the public scrutinize 
a fault tree prepared by a technical expert, discover (what 
seems to them to be) an important omission, and doubt the 
quality of the entire analysis and the competence of the 
analyst (e.g., comments to Barrager, Judd & North, 1976 or 
Birnbaum, Wong & Wong, 1976; Settle & Golden, 1974). 


In contrast with this "credibility" hypothesis, the 
"availability" hypothesis (Tversky & Kahneman, 1973) suggests 
that what is out of sight is also out of mind. Problems that 
are not mentioned explicitly may not be thought of. People 
may not realize what's missing and, therefore, may overestimate 
the completeness of the analysis. Suspecting this to be the 
case, critics of technological projects who discover omissions 
in fault trees have cast doubt not on the competence but on the 
integrity of the analysts, charging the analysts with deliberately 
omitting problems in order to induce the public to underestimate 


the toral risks associated with the project. 


A third hypothesis is that people have such well-defined 
frequency representations in their minds (Howell, 1973) that 
they will appropriately realize what's missing. It is also 


possible that availability and credibility effects might cancel 


one another, producing appropriate “other" judgments. 
Leal. Method 


Stimulus. The basic stimulus for all studies reported 
here is the "car won't start" fault tree of Figure 1-1. It was 
constructed by consulting a variety of shop and repair manuals, 
mechanics and car buffs, attempting to make it as complete as 
possible.t We distinguish between three leve]s of detail: 
Level 1, presenting just the syste names (e.g., "battery 
charge insufficient"); Level 2, listing three to five component 
problems for each system (e.g., 1. Faulty ground connections; 
2. Terminal loose or corroded; 3. Battery weak); and Level 3, 
fuller detail on each of the component problems that could be 
elaborated, that is, everything shown in Figure 1-1. Subjects 
in Experiment 1 were presented with Level 2 detail. 


Design. Four separate groups of subjects participated. 
Two groups received the full, unpruned tree of Level 2. They 
differed only in that the experimenter read aloud the entire 
tree to one group (N = 58); the other group (N = 35) read the 
tree by themselves, without an enforced amount of time for 
examining it. Each of the two other groups was given a 
different pruned tree; one group (N = 29) saw a tree that was 
missing the starting, ignition and mischief branches, while 
the other group (N = 26) saw a pruned tree that lacked the 
battery, fuel and other engine branches. Since the first two 
groups showed no difference in response measures, the self- 
paced read-to-oneself administration was used for the two 


pruned-tree groups. 


Procedure. Before studying the fault tree, subjects 


were told: 


Every day, across the United States, millions of drivers 
perform the act of getting into ar automobile, inserting a key 


in the ignition switch and attempting to start the engine. 


Sometimes the engine fails to start and the trip is 


delayed. 


We'd like you to think about the varicus problems 
that might be serious enough to cause a car to fail to start 


so that the driver's trip is delayed for at least one minute. 


The chart on the next page is intended to help you 
think about this problem. It shows six [three] major 
deficiencies that cause a car's engine to fail to start. 

These major categories probably don't cover all possibilities, 
so we've included a seventh [fourth] category, All Other 


Problems. 


Please examine this diagram carefully and answer the 


following question. 


For every 100 times that a trip is delaye'! due to 
"Starting failure," estimate, on the average, how many of 
those delays are caused by each of the seven [four] factors. 
Make your estimates on the blank lines next to the factors 


named below. Your estimates should sum to 100. 


The numbers above in brackets were given to subjects 


who saw the pruned trees. 


Afcer assigning proportion, all subjects were asked: 


Please answer the following question. Over the next 
1000 times in the U.S. that drivers attempt to start their 
cars, how many times will the drivers exverience delays in 
starting the engine serious enough to delay their departure 


by at least one minute? 
Answer: times out of 1000 


They were also asked a number of questions about the 


extent of their current and past experience with cars. 


Subjects. One hundred and forty-eight persons who 
responded to an advertisement in the University of Oregon 
student newspaper participated. They were assigned to an 
experimental group according to their preference for experiment 


date and hour. 
1.2 Results 


The difference in the mean proportion of problems 
assigned to each branch for the self-paced group and the read- 
aloud group was minimal (mean absolute difference across the 
seven branches was .018). Results for these groups were 


combined. 


If pruned-tree subjects were sensitive to what had been 
omitted in the trees they studied, the proportion of problems 
that they attributed to "other" would equal the sum of the 
proportions of problems attributed to the pruned branches and 


to “other" by subjects who saw the full tree. If the 


availability hypothesis is correct, "other" should be assigned 
a lower proportion in the pruned tree; if the credibility 


hypothesis is correct, that proportion should be greater. 


Table 1-1 presents the mean proportion of startir.j 
failures attributed to each branch and to "other." Pruned-tree 
subjects clearly failed to appreciate what had been left out. 
For pruned group I, “other" should have increased by a factor 
of six (from .078 to .468) to reflect the proportion of failures 
due to starting, ignition and mischief. Instead, “other" was 
only doubied, while the importance of the three systems that 
were mentioned was substantially increased. A similar picture 
emerged with the second pruned tree. Although these results 
clearly favor the availability over the credibility hypotheses, 


subjects did show some sensitivity to what had been omitted. 


The proportion of problems attributed to "other" was significantly 


greater for both pruned-tree groups than for the unpruned-tree 
group (for both t tests, p > .001). In addition, pruned=-tree 
group II, which saw a tree with more important branches deleted, 
assigned a higher proportion of problems to "other" than did 


pruned-tree group I (t = 2.59, df = 53). 


Only one subject of the 55 in the two pruned groups 
assignee to "other" a proportion of problems greater than or 
equal to the sum of the proportions of problems assigned to the 
missing branches (plus "other") by the unpruned-tree group. 
There was no tendency for subjects who rated themselves higher 
in expertise (on a five-point scale) to assign a higher 


proportion of problems to "other" in any of the groups. 


There were no significant differences between the 


pruned- and full-tree groups in the median likelihood that 
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a randomly selected car would not start. This is a bit 
surprising, because the support for the availability explanation 
noted above would suggest that mentioning more specific sources 
of difficulty to unpruned-tree subjects would increase the 
availability of starting failure. Medians were used because of 
the presence of some extremely high estimates (e.g., 600 starting 
failures in 1000 tries) which would have unduly affected the 
means. The relative frequency of such unrealistic estimates 
(even among experienced drivers) suggests that this measure was 
not entirely successful, making results obtained with it somewhat 


dubious. 


lies 3 Discussion 


Pruned-tree subjects clearly did not appreciate how much 
had been left out, and, as a consequence, overestimated the 
exhaustiveness of the branches they saw. One could speculate 
that this effect would be even more pronounced with fault trees 


concerning technical systems less familiar than the present. 


Although the enormity of the present effect suggests that 
it may be quite robust, one might wonder whether it was due, at 
least in part, to unpruned-tree subjects either (a) assuming that 
the experimenters were competent and thus had provided a 
reasonably complete tree or (b) not attending sufficiently to 
the "other" branch. Experiment 2 was conducted to explore these 


possibilities. 


2. EXPERIMENT 2: FOCUSING ON "ALL OTHER PROBLEMS" 


2.1 Method 


Experiment 1 was repeated with several changes. In 

order to feocus subjects" attention on what was left out, a 
paragraph was added to the instructions saying, "In particular, 
we'd like you to consider its completeness. That is, what 
propertion of the possible reasons for a car's not starting 
are left out, to be included in the category 'All Other 
Problems?'" Their proportion estimation task was reduced to 
answering "For every 100 times that a trip is delayed due to 
"starting failure,' estimate, on the average, how many of 
those delays are caused by factors not included in the chart. 

out of every 100 cases would fall in the category ‘All 
Other Problems. '" 


Because of the difficulty Experiment 1 subjects seemed 
to have in assessing the likelihood of starting failure with 
a randomly selected car, that question was expanded by (a) 
asking them not only about a randomly selected car, but also 
about their own car, and (b) asking not only for an absolute 
judgment (how many times out of 1000 attempts), but also for 
a relative judgment (how many times more or less likely is 


starting failure than a flat tire?). 


Eighty-two subjects were recruited as in Experiment 1. 


The fault trees and instructions were read aloud. 


2.2 Results 


Focusing subjects' attention on the "other" branch of 


the unpruned tree had little effect on the unpruned-tree group. 


The mean proportion of starting failures assigned to "other" 

was actually slightly smaller than the proportion in Experiment 1 
(.062 vs. .078; t = 0.71; d£& = 121). It did, however, increase 
by about 50% the proportion of problems that subjects who saw the 


‘ 


pruned trees attributed to "other." For pruned tree I, the 
proportion attributed to "other" increased from .140 to .217 
(t = 1.88, df = 49); for pruned tree I, from .227 to .346 

(t = 1.81; df = 58). In both cases, though, the proportion 
attributed to "other" was still much less than it should be. 


These results are shown in Table 2-1. 


Subjects who received the unpruned tree with its full 
panoply of problems thought that the absolute likelihood of 
a starting failure was greater than did subjects who saw the 
pruned trees (Mann Whitney U Test, z = 2.42 and 1.77, for others' 
cars and own car, respectively). About 90% of the subjects who 
saw the full tree thought that starting failure was more likely 
than a flat tire, compared with about 75% of subjects who saw the 
pruned tree (p < .10 for difference in proportions, both for 
others' car and for own car). Whereas the median pruned~tree 
subject thought tha* a starting failure was 5 times as likely as 
a flat (for either kind of car), subjects who saw the full tree 
thought that starting failure was 20 times more likely for a 
randomly selected car and 60 times more likely for their own car 
(Mann Whitney U Test, z = 2.26 and 4.26, respectively). Seeing 
the full tree apparently made starting failure seem rather more 
likely. 


2.3 Discussion 


Focusing subjects' attention on what is missing improved 


their awareness, but only partially. Because it aid not lead to 


TABLE 2-1 


THE EFFECTS OF FOCUSING SUBJECTS' ATTENTION ON 
"ALL OTHER PROBLEMS" 


Proportion Freguency of Starting Failures 

of problems Other's Car wn Car 

attributed More likely More likely _ 

to “other” than flat Median than flat Median 

mae Should ~ Med. out of Med. out of 
N Observed be % ss Resp. 1000 % Ss Resp. 1000 


Experiment 1 


oe 078 75 
ack “2 -140 . 468 59 
a ane 287 .611 at 

a Experiment 2 - _ 
i“ ae 067 88.9 20 a5 90.4 60 47 
nati 22 249 468 m0 6s 38 83.7 5 35 
oe 34 . 346 -611 75.5 9.6 21.4 7237 XO 20 


Tree IJ 


Note. Subjects in Experiment 2 were instructed to focus on the "other" category. 
Data from Experi.nent 1 are included here for comparison. 
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an exaggerated estimate of what was missing, such focusing would 
appear to be a valuable procedure whenever confronted with a 
fault tree or similar representation. It remains to be seen how 
fault tree designers and evaluators can be brought to a fuller 
appreciation of how adequate a problem presentation is, 
particularly when the missing elements are ones with which they 
are totally or partially unaware. Perhaps a useful rule of thumb 
would be: The proportion of missing sources of trouble is 
proportional to the number of things I can think of that are 
missing multiplied by a measure of my general familiarity with 
the system (where greater familiarity is assigned a lower score). 


In other words, if I don't know much but still can detect 


something missing, then this fault tree representation is quite 


incomplete. 


i a” a, 


a OE, Be 


3. EXPERIMENT 3: LEVEL OF DETAIL 


Another discretionary decision faced by someone 
designing a fault tree for public display is how much detail 
to present. A designer must consider whether additional detail 
serves to inform the viewer or leads to confusion, feelings of 
incompetence or undue apprehension over the large number of 
sources of trouble presented. Experiments 1 and 2 showed that 
omitted branches are essentially out of mind. Experiments 3 
and 4 examined the effect on problem evaluation of exposing 


more or less detail for the displayed branches. 
3d Method 


The car-won't-start fault tree was presented to four 
different groups of subjects with three different levels of 
detail, Levels 1 and 3 described above (system names only and 
full detail) and another called Level 2/plus. This intermediate 
level included everything in Level 2 along with full (Level 3) 
detail for just one branch. Subjects receiving such a fault 
tree were told that a similarly detailed analysis could be 
performed for other branches. One group at Level 2/plus 
received full detail on the fuel system; another received full 


detail on the battery system. 


One hundred and ten new subjects were recruited as in 
Experiments 1 and 2. Questionnaires were distributed in a group 


setting and subjects read them at their own pace. 


In this and all subsequent experiments, the proportion 
estimation task of Experiment 1 was u ed (subjects gave proportions 


to all Level 1 categories), while the four final questions from 


Experiment 2 were used (number of failures out of 1000 and 


comparison with a flat tire for both other's car and own car). 


3.2 Results 


The upper half of Table 3-1 presents results from 
Experiment 3, along with the results for the corresponding 
group given Level 2 in Experiment 1. The effects of the various 


manipulations of detail seem quite modest. 


One might expect that when just one system receives 
more detail than the others, as in the two Level 2/plus groups, 
that system's proportion of problems would be elevated. However, 
neither battery nor fuel, when given a greater detail, received 
a proportion significantly (a = .05) larger than these systems 
received in the other groups. 


Similarly, one might expect a lower proportion of problems 
attributed to mischievous acts at Level 3, when it is the only 
branch without additional detail (see Figure 1-1), than for the 
other detail levels. No such effect was found. 


The battery and fuel systems, which had more detail at 
Level 3 than did the other systems, were judged more likely by 
that group than by other groups (for battery, t = 2.95 compared 
with Level 2 and t = .86 compared with Level 1; for fuel, 
t = 1.14 compared with Level 2 and t = 2.49 compared with 
Level 1; df = 70 in all tests). 


Finally, the proportion of problems attributed to "All 
Other Problems" decreased monotonically as level of detail 


increased (for linear trend; F = 7.40; df = 1,141; p < .01). 


TABLE 3-1 


EFFECT OF PRESENTING VARYING DEGREES OF DETAIL 


Mean proportion of starting failures attrihuted to: 


Median # Starting 
Failures in 1,000 


Level of Detail N Battery Starting Fuel Ignition Engise Mischief Other (others! cars) 
‘ Experiment 3--Self Paced 
System Names Alone (Level 1) 35 303 -162 149 -156 062 052 116 112 
Minimal Detail (Level 2)" 35 231 205 .194 ae 083 046 -092 100 
One Branch with Full Detail 
Battery 19 284 - 184 .182 .180 066 .039 .065 100 
Fuel 19 «224 256 Ba .140 .067 051 . 088 100 
Full Detail (level 3) yi 337 126 . 232 Pe i as -061 .057 062 70 
Experiment 4--Read Aloud 
System Names Alone (Level 1) 35 330  K52 . 136 172 067 -060 083 27 
Minimal Detail (level 2)° 58 . 284 .189 192 139 072 054 -970 Te 
One Branch with Full Detail 
Battery 22 347” 185 ols .124 052 051 -066 5a 
Starting 26.260 aa” aaa. 08 062.095 50 
Fuel 23.304 ie “0? ter ees 90 
Full Detail (Level 3) 67 294 -198 - 186 | .080 -038 053 50 


= Unpruned self-paced group from Experiment 1 
. Single branch presented in full detail 


S Unpruned read-aloud group from Experiment 1 


a=3 


Despite the expectation that the probability of starting 
failure would increase as the level of detail increased, none of 
the four measures of this probability showed this effect. 
Indeed, the only such measure shown in Table 3-l, starting 


failures out of 1000 for others' cars, showed a nonsignificant 


trend in the opposite direction. 


4. EXPERIMENT 4 


Before concluding that detail has only a modest effect, 
we explored the possibility that mode of presentation mattered 
here (although it had not in Experiment 1). We spesulated that 
subjects presented the rather voluminous fault trees for 
Levels 2/plus and 3 may have skimmed over them, not attending 
to the greater detail they offered. Experiment 4 replicated 
Experiment 3 with fault trees read aloud to 173 new subjects. 

A third Level 2/plus group was added, giving full detail on 
the starting system. 


4.1 Results 


The lower portion of Table 3-1 presents the results from 
Experiment 4, along with the appropriate Level 2 results from 
Experiment 1. With Levels 1 and 2, the differences between 
Experiments 3 and 4 were negligible. The mean absolute 
differences in proportions for the seven branches were .016 and 
-018, respectively. With Level 2/plus, there was somewhat more 
of a difference. As a highlighted branch, the battery system 
increased from .284 in Experiment 3 to .347 in Experiment 4 
(t = 1.33; df = 39); the fuel system, however, changed little 
(.174 vs. .184; t = 0.34; df = 40). 


Unlike Experiment 3, here the battery system received 
a somewhat (but not enormously) greater share of attributed 
problems when it was highlighted (Level 2/plus) than in all 
other conditions. So did the starting system; however, th 
fuel system again failed to show this effect. 


Unlike Experiment 3, mischievous acts did show a 
significantly smaller proportion of problems at Level 3, where 
it is the only branch lacking full detail (z = 2.95 compared 


with Level 1; z = 2.17 compared with Level 2). 


Again in contrast to Experiment 3, the battery and fuel 
systems, the ones with the greatest detail, were not more likely 
in Level 3, where this detail is most apparent, than in other 


levels. 


The proportion of problems attributed to "All Other 
Problems" in Experiment 4 showed the same tendency to decrease 
as detail increased found in Experiment 3, except for one 
reversal: The weighted mean across the three Level 2/plus groups 
was .080, rather than a value between .053 and .070. Despite the 
one reversal, the linear trend in a one-way ANOVA was significant 
(FP = 4.44; Gf = 1,227; p < .05). 


In Experiment 4, the median starting failure rate for 
others' cars was similar for Levels 1 and 2 (Mann Whitney U 
Test, z = .56), but somewhat higher for Levels 2/plus and 3 
than for Levels 1 and 2 (Mann Whitney U Test, z = 1.83). No 
such differences, even marginally significant, were observed for 
“one's own car" (no shown), suggesting that subjects' judgments 
regarding their own cars were less susceptible to this 
manipulation. Subjects in Experiment 4 also showed some effect 
due to detail on the likelihood of starting failure compared with 
that of a flat tire for both their own and others' cars. : r 
Level 1, roughly 75% of subjects thought that starting failure 
was more likely; fur Levels 2, 2/plus a4 3, approximately 90% 
thought so. The difference in proportions was significant 


(z = 2.12 for others' cars, z = 2.47 for own car). 


4s 2 Discussion 


Compared with the enormous availapility effect found in 
Experiments 1 and °, the present results are modest. Only one 
of several hypothesized effects due to variation in detail was 
found in both Experiments 3 and 4: The proportion of problems 
attributed to "All Other Problems" decreased as detail increased. 
Further research with greater power is needed to explore the 
other effects, found in one but not both experiments. Even if 


real, these possible effects are probably small. 


One must conclude that amount of detail did not produce 
large changes in people's perceptions. The mere mention of 
a branch (Level 1) appeared to allow subjects to make a fairly 
accurate estimate of how troublesome that branch would look 
when fully detailed. Level 2/plus subjects were somewhat 
(although not completely) successful in compensating for the 
detail missing on the more minimally presented branches. The 
relatively small effect of increased detail on estimated 
starting failure rate suggests that subjects given little detail 


realize their own ignorance. 


5. EXPERIMENT 5: SPLITTING AND FUSING BRANCHES 


A third area in which the designer of a fault tree must 
make discrelionary decisions is in organizing the various 
sources of trouble into branches. Although functional 
relationships introduce some constraints, the designer must 
make decisions like whether "disruptions of wiring" belongs 
under "mischievous acts" or "ignition system defective." 

A common dilemma will be broad versus narrow categorization. 
"Starting system defective" and “ignition system defective" 
could be lumped together. “Ignition system defective" could 
be split into "ignition system defective" (items 1 and 3 in 
‘Figure 1-1) and "distribution system defective" (items 2 and 
4). Experiment 5 examines the effect on fault-tree evaluation 
of the way in which a fixed amount of information is organized, 


specifically, whether broader or narrower categorization is used. 


Sak Method 


The effect of breadth of categorization on fault-tree 
evaluation was studied by creating four new versions of the 
Level 2 fault tree (Figure 1-1). Two were created by splitting 
different existing branches into two branches containing 
(between them) the same information. In one, "ignition system 
defective” was split into "ignition system defective" with 
items 1 and 3 (coil faulty and spark plugs defective) and 
"distribution system defective" with items 2 and 4 (distributor 
faulty and defective wiring between components); in the second 
version, "fuel system defective" was split into "fuel system 
defective" (items 1 and 2) and "“carburetion defective" (items 3 
ard 4). Thus, these two trees each had eight branches (counting 
"other"). Two additional trees were created by fusing two 
branches of the full tree. In one, "Starting system defective" 


and "ignition system defective" were combined into one branch; 


in the second, “fuel system defective" and “other engine problems" 
were combined. Thus these fused versions each had six branches 


(counting "other"). 


One hundred and fifteen subjects were recruited as before. 
Each subject saw only one tree. The tasks for the subjects were 


the same as in Experiments 3 and 4. The trees were read aloud. 
Sine Results 


Table 5-1 presents the proportion of starting failures 
attributed to each of the manipulated branches, along with 
comparable data from Experiment 1. In every case, a set of 
problems was perceived as more important when it was presented 
as two branches than when presented as one. The mean increase 
over the four cases was .066; in general a set of problems was 
attributed about a third greater portion of the total number of 
starting failures when presented as two branches. The number 
of subjects given 8-branch trees wno assigned the two new 
branches a combined proportion higher than the mean assigned by 
subjects in Experiment 1 to the c:mparable single branch was 20 
out of 27 (sign test, p = .010) for the fuel-syst« nm-split group 
and 19 out of 26 (p = .014) for the ignition-system-sphit group. 
The number of subjects who assigned a proportion to the new fused 
branch lower than the corresponding mean sum in Experiment 1 was 
24 out of 33 (p = .007) for the starting-ignition-combined group 
and 23 out of 29 (p = .001) for the fuel-engine-combined group. 


There were no differences in either the relative (compared 
to a flat tire) or absolute likelihocd of starting failure for 


own or others' cars, for subjects seeing the information presented 


in 6- or 8-branch fault trees (not shown). 


TABLE 5-1 


EXPERIMENT 5: EFFECTS OF SPLITTING AND FUSING BRANCHES 


Separate Together 


Mean proportion of Mean proportion of Difference® 
Starting Failures Scum Starting Failures 


Splitting Existing Branches 


Fuel -161 
Carburetion -099 
(N=27) 
Ignition - 082 
Distribution Ae lak 
(N=26) 


Fusing Existing Branches 


Starting ~195 248 
Ignition ~144 (N=33) 


Fuel -193 213 
Other Engine -076 (N=29) 


Note. In all places where the number of subjects is not indicated, the results 
are those from the unpruned tree (Level 2) subjects in Experiment 1 
(Table 1-1), for which N = 93. 


separate Sum Minus Together 


Discussion 


Experiment 5 showed that the more pieces into which 
a system of failure pathways is organized, the more important 
that system seems. One possible explanation is that people tend 
to assign some minimum probability to any category with which 
they are faced. A branch split in two receives two portions of 
this minimum probability, either because the smallest non-zero 
estimate allowed by the response mode was .01, or because of an 
assumption that "if they decided to include this branch, it must 
have some minimal import." An indirect way to evaluate this 
hypothesis is to estimate what proportion of problems is 
attributed to a category of "minimal import." For the 93 subjects 
in Experiment 1 who saw the unpruned Level 2 tree, the mean 
proportion of problems assigned to the least important branch 
was .033; the mean lowest proportion excluding zero responses 
was .040. (Only 11% of subjects ever assigned a zero proportion 
of problems to any branch.) By this criterion, the increased 
importance (.066) garnered by splitting a branch was about 
twice that which might be attributed merely to increasing the 


number of categories. 


EXPERIMENT 6: EXPERTS 


Our discussion so far has been concerned with lay 
persons' evaluations of fault trees presented to them by 
technical experts. One might wonder, though, whether the 
effects we have found also affect the technical experts 
themselves. Fischhoff (1977) lists many incidents in which 
experts designing fault trees for important technological 
systems were apparently unaware of major omissions, and 
therefore greatly overestimated the exhaustiveness of their 
own analyses. Experiment 6 examines whether that which is out 


of sight is also out of mind for technical experts. 


Experiment 6 replicated Experiment 1 using as subjects 
experienced mechanics in Eugene, Oregon. Thirty copies of 
pruned tree I (lacking the starting system, ignition system and 
mischief branches) and thirty copies of the unpruned tree 
(Level 2) were distributed to experienced mechanics at six 


major garages. 


In the accompanying letter, they were told: 


We are asking poeple like yourself who work with cars 
to give us their opinion about certain types of engine starting 
problems. We hope you will agree to read the questionnaire and 


give us your opinions. 


Actually, we are not just interested in automobile engines. 
We're concerned with all kinds of complex mechanical systems 
ranging from automobile engines to nuclear reactors. In 
particular, we're interested in the ways these systems break 


down and the judgments about these breakdowns made by people who 


repair these systems. That's why we're asking for your opinions. 


If you are willing to participate in this study, please 
read the instructions on the next page; then fill out the two 
short questionnaires and mail them to us in the enclosed, 


addressed and stamped envelope. We'll mail you a check for $3. 


There are no "right" answers to these questions. At 
least we don't know the answers. That's why we're interested 


in your opinions. 


Please work alone and don't refer to any books or discuss 


the questionnaire with anyone before you have completed it. 


Otherwise, the questionnaire was like that in Experiment 1. On 
the final page, they were asked to iu.dicate (i) the rate of 
starting failures per 1000 attempts (for drivers in the U.S.); 

(b) the number of years they had made all or part of their living 
working with cars; and (c) how knowledgeable they were about 
automobile engine problems compared to other mecnanics: below 


average, average, above average, or much above average. 


6.1 Results 


Twenty nine of the 60 questionnaires were retuzned, 14 
from the full-tree group and 13 from the unpruned-tree group. 
Two mechanics rated themselves as much above average in 
knowledge, 18 as above average, 8 as average and one as helow 
average. They had from 2 to 43 years of experience, with the 
mean for the full-tree group being 12.2 years, and for the 


pruned-tree group 19.8 years. 


Table 6-1 presents the mean proportion of starting 
failures attributed toa each branch by subjects iu the present 


experiment as well as the unpruned-tree resuli:s from Experiment 1 


TABLE 6-1 


EXPERIMENT 6: JUDGMENT OF EXPERTS 


Mean Proportion of Starting Failures Attributed To: 


: Median #¢# 
Starting 
Starting Fuel Ignition Failures 
N Battery System System System Engine Mischief Other n 1000 
Unpruned 
Tree 
Ordinary j 
z a ~U85 2153 z a -05 - ; 5 
Subjects 93 264 195 19 144 076 051 O7E 75 
Unpruned 
Tree 
Experts 13 -410 -108 - 096 .248 “O'S. 025 060 20 
Pruned 
Tree I 
Experts 16 -483 ---- aes oor .073 ~—<- ~215 100 


@ should be .441 
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for the sake of comparison. The experts thought that battery 


and ignition were more serious problems than did the regular 
subjects, and that the starting and fuel system were less 
important. They more or less agreed about the completeness 


of the tree, assigning .060 to "other." 


The combined proportion of problems attributed to the 
branches deleted from the pruned tree (plus "other") was .441. 
Pruned-tree subjects assigned to "other" a mean proportion of 
only .215 (Mann Whitney U Test, z = 2.87). The respective 


median responses were .47 and .16. 


Within the pruned-tree group, the two subjects who rated 
themselves as much above average in knowledge assigned a mean 
to "other" of .075, the nine above average mechanics assigned 
-278, and the five average ones .158. There was a rank order 
correlation (tau) of .058 between number of years of experience 
and proportion assigned to "other." Thus, neither self-rated 
degree of knowledge nor actual experience had any systematic 
relation to ability to detect what was missing from the tree. 


— 


7. GENERAL DISCUSSION 


The most dramatic result of these studies was subjects' 
inability to appreciate how much had been omitted from the 
pruned fault trees. Exaggeration of fault tree completeness 
was found with both "regular" subjects and experienced mechanics. 
The fact that omission of major branches triggered only minimal 
awareness of the inadequacies of the pruned tree lent strong 
support to the availability hypothesis over the credibility 
hypothesis, as did the modest improvement when subjects' 


attention was focused on completeness. 


How might things that are out of sight also be out of 
mind? One obvious reason is ignorance. There is no way to 
consider something which one has never heard of and which is 
not mentioned. In a discussion of the omissions which seem to 
plague technical experts performing formal risk assessments, 
Fischhoff (1977) suggests several other reasons: (a) failure 
to consider the imaginative ways in which human error can mess 
up a system (e.g., the Brown's Ferry fire in which the world's 
largest nuclear power plant almost melted down due to a 
technician's checking for an air leak with a candle in direct 
violation of standard operating procedure); (b) insensitivity to 
the assumptions an analysis makes about constancies in the world 
in which the system is embedded (e.g., no major changes in 
government regulatory policy); (c) overconfidence in current 
scientific and technological knowledge (i.e., assuming that 
there are no new chemical, physical, biological or psychological 
effects to be discovered); and (d) failure to see how the system 
functions as a whole (e.g., a system may fail because a back-up 


component has been removed for routine maintenance). 


em emtemnen sat 


Although similar problems seem likely to afflict the 


designers and viewers of fault trees, this list is probably 
incomplete. Expanding and validating such a list is important 
not only for our understanding of how people conceptualize 
complex, fallible systems, but also for helping them bet:ter 
describe and comprehend such systems. Focusing on "other" 

was somewhat, but only partially, successful in helping people 
appraise the completeness of a fault tree. Improved 
understanding of the reasons for omissions will help in 
designing better debiasing procedures, ways to approach an 
evaluation task which provides a more veridical perspective. 
Where such "cognitive engineering" seems ineffective, people may 
need rules of thumb like the suggested rule for estimating how 
many sources of trouble one can't think of on the basis of how 
many one can think of and one's familiarity with the problem area. 


Because of the importance of intelligent public 
participation in debates about technology (Casper, 1976; Slovic, 
Fischhoff & Lichtenstein, 19°45), these are critical issues to 
which psychologists might address themselves. Such research 
does, however, raise serious ethical questions because of the 
possibility that the results will disclose ways in which public 
opinion can be manipulated. For example, Experiments 1 and 6 
suggest that one can get the public to focus on those issues one 
thinks are important by never mentioning other issues. Even our 
debiasing procedure (focus on “other") in Experiment 2 was a form 
of manipulation, changing people's perceptions from what they 
would otherwise be. (Although we believe that this change is for 
the better, the point is moot as long as we do not know the 
proportion of problems in fact due to each branch.) Perhaps the 
prime responsibility of the discoverer of such effects is to 


ensure that they receive the broadest possible dissemination, 


cay, 


so that both potential manipulators and the potentially 


manipulated are aware of them. It may turn out that we 
psychologists are merely discovering "tricks" that manipulators 
have known about all along. Such research can also suggest 
aspects of presentation which may have relatively little effect 
on people's judgments (varying level of detail), atteinpted 
manipulations which can be readily overcome (providing more 
detail for one branch) and manipulations whose effects, although 


consistent, may be too small to have applied implications. 


Aside from calling for obviously needed constructive 
replications (different trees, different subjects, different 
experts, etc.), these results suggest a variety of questions 
for future study: Would credibility be a stronger effect than 
availability if pruned fault trees were presented by distrusted 
technocrats rather than moderately trusted experimental 
psychologists? What happens when people construct their own 
fault trees? What happens when the detailed items from a 
fault tree are merely listed rather than being organized into 
categories? What happens with trees dealing with more technical 
areas, or with non-technical areas for which the assessors' 
feelings of competence vary? What happens when minor rather 
than major items are deleted? What if an omitted item is 
somehow brought to the subjects' attention (as often happens 
in public debates)? What individual differences, if any, 
interact with these aspects of presentation in affecting 
evaluations (as mentioned, the obvious covariate of years of 
experience was uncorrelated with the mechanics' awareness of 
how much had been deleted from the pruned tree)? What if 
detail is manipulated not by adding facts, but by adding 
flourishes, by "fleshing out" possible sources of trouble with 


vividly detailed scenarios describing just how they might happen? 


Why were subjects' perceptions of the overall starting failure 


rate relatively impervious to our manipulations? Was it merely 


a problem of measurement? 


These studies have both supported the notion that 
availability affects frequency estimates and helped clarify 
how availability mechanisms work. Failure to mention a 
possibility had a major effect, increasing the detail of what 
was mentioned had a small effect, and the packaging of what is 
mentioned alsu made a difference. Experiment 2 showed, as did 
Beyth-Marom and Fischhoff (1977), that forcing people to work 
their memories (or imaginations) harder can improve their 
likelihood judgments somewhat. Clearly, much more work is 
needed on the retrieval, perception and representation of 


frequency information. 


8. FOOTNOTES 


1 
To our chagrin, the first starting failure encountered by 


anyone connected by this study was not included in our fauit 
tree: an ignition key not turning because the steering wheel 


lock had caught the ignition switch. 


o=i 
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